HAL 

archives-ouvertes 


Vision based control for Humanoid robots 

Claire Dune, Andrei Herdt, Eric Marchand, Olivier Stasse, Pierre-Brice 

Wieber, Eiichi Yoshida 


► To cite this version: 

Claire Dune, Andrei Herdt, Eric Marchand, Olivier Stasse, Pierre-Brice Wieber, et al.. Vision based 
control for Humanoid robots. IROS Workshop on Visual Control of Mobile Robots (ViCoMoR), Sep 
2011, San Francisco, USA, United States, pp.19-26. hal-00639681 


HAL Id: hal-00639681 
https: //hal.inria.fr/hal-00639681 

Submitted on 9 Nov 2011 


HAL is a multi-disciplinary open access 
archive for the deposit and dissemination of sci¬ 
entific research documents, whether they are pub¬ 
lished or not. The documents may come from 
teaching and research institutions in France or 
abroad, or from public or private research centers. 


L ’archive ouverte pluridisciplinaire HAL, est 
destinee au depot et a la diffusion de documents 
scientifiques de niveau recherche, publies ou non, 
emanant des etablissements d’enseignement et de 
recherche frangais ou etrangers, des laboratoires 
publics ou prives. 



IROS Workshop on Visual Control of Mobile Robots (ViCoMoR) 
September 26, 2011, San Francisco, California, USA 


Vision based control for Humanoid robots 


C. Dune, A. Herdt, E. Marchand, O. Stasse, P.-B. Wieber, E. Yoshida 


Abstract —This paper presents a visual servoing scheme 
to control humanoid dynamic walk. Whereas most of the 
existing approaches follow a perception-decision-action scheme, 
we hereby introduce a method that uses the on-line information 
given by an on-board camera. This close looped approach allows 
the system to react to changes in its environment and adapt 
to modelling error. Our approach is based on a new reactive 
pattern generator which modifies footsteps, center of mass and 
center of pressure trajectories at the control level for the center 
of mass to track a reference velocity. In this workshop, we 
present three ways of servoing dynamical humanoid walk : 
a naive one that compute a reference velocity using a visual 
servoing control law, a second one that takes into account the 
sway motion induced by the walk and an on going work on 
vision predictive control that directly introduces the visual error 
in the cost function of the pattern generator. The two first 
approaches have been validated on the HRP-2 robot. These 
close loop approaches give a more accurate positioning than 
the one obtained when executing a planned trajectory especially 
when rotational motion are involved. 

I. INTRODUCTION 

Humanoid robots are designed for human environments, 
defined as unstructured and dynamic environments [1] where 
objects move outside robots’control. In order to complete a 
specific task, humanoid robots must perceive and react to er_* 
vironmental changes. Vision based control may help them to 
perceive their surroundings in order to adapt their behaviour 
efficiently. Indeed, most of the humanoid robots are equipped 
with cameras that provide rich information without adding 
so much weight and size. The use of embedded camera is 
attractive because it avoids equipping the environment with 
additional sensors, and thus the system is more autonomous. 
Yet, extracting data from these cameras is a real challenge, 
especially while walking. 

In this paper, we introduce a monocular visual servoing 
scheme to control the HRP-2 walk towards an object with 
taking into account the peculiar motion of the on-board 
camera induced by the stepping. 

A. State of the art 

Previous works on humanoid walking control assume that 
the robot path is defined before computing the actual joint 
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control to realize it. They generally follow a perception- 
decision-action scheme: first, a sensor acquires data on the 
world and/or the robot state, then, suitable footsteps over a 
time horizon are decided, and the trajectories of the center 
of mass (CoM) and the center of pressure (CoP) are com¬ 
puted while respecting the stability constraints. Finally, the 
control of the legs is computed by inverse kinematics. This 
perception-decision-action loop has proven to be fast enough 
to realize impressive demonstrations for stair-climbing and 
obstacle avoidance [2], [3], [4], [5]. 


k M 0 



Fig. 1. The robot has to reach a desired position with regards to an object 

We claim that a visual servoing control scheme is well 
suited for vision based walking motion generation because 
it compensates for model errors. Visual servoing proved 
to be successful for grasping tasks with standing [6], [7] 
or walking humanoids [8], [9]. In [8], visual servoing is 
used to control a humanoid avatar along landmarks. The 
upper body is approximated by the kinematic chain that 
links an on-board camera to the CoM. The lower body is 
controlled by adding two translational degrees of freedom 
to the CoM. The translational velocity of the CoM is sent 
to a kinematic locomotion module which control the legs 
motion. In [9] a whole body visual servoing scheme based 
on a hierarchical stack of task is introduced. However, the 
footsteps are predefined. The leg motion is thus set to be 
the task of higher priority. Therefore visual-servoing in this 
context is projected in the null-space of the pre-defined 
walking path. On the contrary in this work, the controller 
driving the walk is directly guided by vision. 

Few work deal with footsteps, CoM and CoP trajectories 
modification inside the preview window. The work presented 
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in [10] shades some light on this problem. It shows that 
modifying the next landing position of the flying foot might 
impose a new CoP trajectory going out of the support 
polygon. This can jeopardize the equilibrium of the robot. 
To solve the problem, the stepping period may be modified 
to reduce this instability [10], at the cost of slowing down 
the robot. A recent method proposes to modify the footsteps 
according to a perturbation applied to the CoP [11]. In the 
current paper the desired velocity computed by a visual 
servoing based controller is directly used to change footsteps, 
while ensuring walking stability constraints and with time 
intervals of constant length. Another difference lays on the 
fact that, on the one hand, the CoP is constrained at the 
center of the footprints, and on the other hand the CoP can 
move freely inside the support polygon. 

B. Contribution 

Our approach is based on a new pattern generator (PG) 
that has been proposed by Herdt et al [12], [13]. It computes 
a reactive stable walking motion for the CoM to track an 
instant reference velocity without predefined footsteps. This 
paves the way to reactive walking motion based on current 
environmental perception. 

In this paper, we introduce a real time vision based control 
of HRP-2 walking motion. It is based on the visual servoing 
scheme we introduced in [14] applied to a positioning task. 
This scheme has the advantage of handling both fixed and 
mobile object. The only requirement is to know at least 
partially the 3d model of the object : some 3d edges for 
angular objects or the diameter of a sphere for a ball. 

C. Paper overview 


1) Motion of the Center of Mass: Let us consider a frame 
C attached to the position of the CoM of the robot and to 
the orientation of its trunk. The position and orientation of 
this frame will be noted c = [c x c y c z c^ c^ c 9 ] , 

with Cardan angles c^, c^ and c e . 

The acceleration c of this frame has to be continuous for 
being realized properly by usual actuators. We will consider 
here that it is in fact piecewise linear on time intervals of 
constant length r, with a piecewise constant jerk "c (third 
derivative of the position) on these intervals. The trajectory 
of this frame over longer time intervals of length nr can 
be simply obtained by integrating over time the piecewise 
constant jerk together with the initial speed c and acceleration 
c. For any coordinate a G {x,y, this leads to 

simple linear relationships 

c ?+1 = s p c a* + U p Ci, (1) 

C?+i=S v c?+U v Ci, (2) 

C?+l=Sa% + U a Ci, (3) 

where the initial state is cf = \c a (ti) c a (U) c a (ti)] T , 

and Cf +i is the vector of the state on the prediction horizon 
that can is defined by 

v*(*i+i)l r c a (ti^y 

riot _ • _ 

°i+l — : 5 • • • Oi+1 — ; 

C a {ti +n )_ _c"(i i+n )_ 

The matrices Z 9 introduced here follow directly from 

recursive application of the dynamics (details on matrices can 
be found in [12]), let T be the sampling period, and N the 
length of the time horizon. The matrix related to the position 
prediction are : 


Section II is dedicated to the new PG description. Section 
III presents the model based tracker and the visual servoing 
control and Section IV presents the results obtained using 
our approach with regards to the execution of a planned 
trajectory. Section V draws the conclusion and perspectives. 
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the one related to the velocity prediction on time horizon are 


II. Prediction Control Scheme for Reactive 
Walking Motion 

This section presents the on-line walking motion generator 
introduced in [12], [13]. The robot is modelled as a linear 
inverse pendulum which fits fairly well with the HRP- 
2 distribution of mass. The control is based on a Linear 
Model Predictive Control scheme that computes the footsteps 
and the optimal jerk of the point mass model to minimise 
the difference between a reference CoM velocity and the 
previewed one. 
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and the one related to the acceleration prediction on time 
horizon are : 


(0 0 1\ 

S a = \ \ \ U a = 

Vo o 1/ 


T 0 O' 

: 0 
1 + N)T ... T 


A. Systems Dynamics 

The humanoid robot is modelled as an oriented mass point 
centred on the robot CoM. This paragraph describes the 
dynamics of a stable walking motion. 


2) Motion of the Center of Pressure: The position z of the 
Center of Pressure (CoP) on the ground can be approximated 
by considering only the inertial effects that are due to the 
translation of the CoM, neglecting the other effects due to 
the rotations of the different parts of the robot. This proves to 
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be a very effective approximation, which leads to the simple 
relationships: 

= cf- (C? - Z*)c?/g, and z? = c? - (cf - zft&Jg, 

where the difference c\ — zf corresponds to the height of 
the CoM above the ground, and g is the norm of the gravity 
force. We will consider here only the simple case where the 
height of the CoM above the ground is constant. In that case, 
we can obtain a relationship similar to (l)-(3): 

Zf+ 1 = S z cf + U z Ci and Z v i+1 = S z c\ + U z cl 
with S z = S p - (c- - z*)S a /g, 

U Z = U P - (cl - zl)U a /g. 

3) Foot step generation: Basically, humanoid nominal 
walking cycle can be divided into two stages: a double 
support phase, where the two feet are on the ground and 
a single support phase, where only one foot is firmly on the 
ground on the other one is flying from its previous position 
to the next one. In this paper the stepping period is set to 
be 800ms with a double support phase of 100ms and single 
support phase of 700ms. 

The new pattern generator selects on-line the feasible 
footsteps on the preview window with regards to the robot 
mechanical properties [15]. Let note F i+ 1 the vector of the 
footstep position on the time horizon. The position of the 
footsteps is then used twice: first to ensure the stability 
constraints on the CoP trajectory and secondly to be included 
in the cost function to attract the CoP trajectory towards the 
center of the polygon of support. 

B. Constraints definition 

To be stable, the dynamics control of the walking motion 
must comply with the following stability constraints. 

1) Constraints on the CoP : since the feet of the robot 
can only push on the ground, the CoP can lie only within 
the support polygon, that is the convex hull of the contact 
points between the feet and the ground [16]. Any trajectory 
not satisfying this constraint cannot be realized properly. This 
needs to be taken into account when computing a walking 
motion with the MPC scheme (4). The foot on the ground 
is assumed to have a polygonal shape, so that this constraint 
can be expressed as a set of constraints on the position of the 
CoP which are linear with respect to the position of the foot 
on the ground but nonlinear with respect to its orientation. 

2 ) Constraints on the foot placement: we need to assure 
that the footsteps decided by the above mentioned algorithm 
are feasible with respect to maximum leg length, joint limits, 
self-collision avoidance, maximum joint velocity and similar 
geometric and kinematic limitations. In order to keep the 
Linear MPC structure of the algorithm, simple approxima¬ 
tions of all these limitations are expressed in the form of 
linear constraints defined in [15]. 

C. Following a reference velocity 

This section sets the optimisation problem to solve to 
ensure that the CoM velocity tracks a reference velocity. 


In order to keep the constraints linear, the optimisation is 
split in two steps: first, translations are treated, then rotations 
along the vertical axis are considered. This control is used 
as the highest priority task in a general inverse kinematics 
framework to compute whole body motion. 

1) Translational velocity: It has been proposed in [12] to 
generate walking motions by directly following a reference 
velocity C*. Only horizontal translations were considered. 
Secondary objectives were also introduced to help obtaining 
a more satisfying behaviour: centring the position of the feet 
with respect to the position of the CoP, and minimizing the 
jerk c (t) to slightly smoothen the resulting trajectory. 
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where C is the mean speed of the CoM over two steps. Intro¬ 
ducing the vector ui = [c* Ff +1 c\ Ff +1 \ of motion 
parameters which automatically computed, this optimization 
problem can be expressed as a canonical Quadratic Program 
with the aforementioned constraints [12]. 

2) Following a reference rotational velocity: If the robot 
trunk has to rotate, then the orientations of the feet have to 
be adapted properly. Yet, introducing 0 as a variable in II- 
B.2 would result in non-linear constraints. In order to keep 
the linear form, Herdt et al [12], [13] chose to predetermine 
the orientation of the feet before solving the translational 
Quadratic Program. 

To increase the robustness of trunk rotational motion, the 
feet orientations have to be aligned with the trunk orientation. 
Furthermore, feet and trunk acceleration and velocity have 
to be limited to avoid infeasible trajectories. This leads to 
the formulation of a decoupled Quadratic Program: 


mm 


\\cf +1 - F? +1 \\ 2 + i\\Cf +1 - 


y i+l I 


Ilif+’i -if+iH < t 

lire _ r e II ^ n FT 

ll^i+l ^i+lll ^ u max 

\\F° +1 -c° i+1 \\<eZ x 

\Wi+l ~ Cf+ill < 


(5) 

(6) 

(7) 

( 8 ) 
(9) 

( 10 ) 


The two terms of the above objective ensure that the trunk 
follows the desired rotational velocity and that at the same 
time the feet are aligned as much as possible with the trunk. 
The constraints assure the feasibility of the desired motions. 


D. Over-all behaviour of pattern generator 

In order to compute a proper control law for the walk, 
we have to understand the over-all behaviour of the pattern 
generator. The PG ensures that the CoM tracks a reference 
velocity yet on average and in the limit of the dimension 
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of the robot (length of legs, actuator torque limit, etc.). We 
describe here these two aspects of the PG (see Fig. 2). 



Fig. 2. The pattern generator ensures that the input velocity is tracked on 
average on the preview horizon. The output of the Model Predictive Control 
is the first control computed on the preview horizon. The difference between 
the reference velocity and the real velocity is mostly a sway motion due to 
the stepping. 

1) Limiting the velocity: In order to ensure the tracking 

of the reference velocity, the three velocity components have 
to be limited to feasible ones, i.e. velocities that respect 
the walking constraints which mainly depends on the robot 
geometry and actuators capabilities. It can be shown that 
the maximum speed for the HRP-2 robot is : cumit = 
(0.2 0.2 0.2) for the considered PG [13]. 

2 ) Sway Motion: In most of the existing PG, the stepping 
motion induces a lateral sway motion that prevents the CoM 
velocity from following instantaneously the expected one. 
The sway motion is mandatory for a proper walk and the 
control law should not compensate for it but cancels its 
effects on the visual error computation. 

Let us define b the additional sway of period T = T step /r. 
Let assume that b is such that b(t)dt = 0. Then the 
behaviour of the PG can be approximated by 

6 = c + b (11) 

where c is the velocity of a virtual average CoM that 
corresponds to a displacement without the stepping. The 
camera velocity can then be written : 

k = k + k V c b (12) 

where c Vk is the twist matrix related to the camera-center of 
mass transform C Mk (see Fig 1). 

III. Vision Based Control 

In this section, a position based visual servoing scheme 
is introduced to compute the velocity that is given as a 
reference to the reactive PG. 

When the robot walks, its stepping makes its head shake 
and oscillate. Each time a foot hits the ground, the impact 
propagates to the robot’s head and the camera jolts which 
causes blur and shift in the image. Moreover, the inherent 
sway motion disturbs the control law. It makes the use 
of on-board images challenging. We will first describe a 
model-based tracking [17] that is robust enough to track a 
known object in such a difficult image sequence. Then, we 


present our visual control law that modify online the current 
measurement to cancel the sway motion. 

A. Model Based Tracking 

The model-based tracking introduced in [17] provides a 
robust solution to the challenging issue of tracking an object 
while walking. It can be used to track geometrical shapes 
(lines, cylinders, ellipsoids, ...) as soon their perspective 
projection can be computed. It estimates on-line the position 
of a known object in the camera frame k M 0 . This tracking 
algorithm can be divided in two steps: i) 2D tracking where 
contour points are locally tracked and ii) pose estimation that 
is based on a non linear iterative algorithm. 

Fig. 3 depicts the tracking principle. 1) Starting from an 
initial pose, the lines of the 3D object model are projected 
on the image and sampled (light blue lines and black points). 
Then the normal to the line are computed for every sampled 
points (yellow lines). Pixels are tested in the neighbourhood 
of the sampled points and along the normal to find the 
maximum gradient response (red points). 2) In a second step, 
a virtual visual servoing is used to find the object position 
by controlling a virtual camera so that the projection of the 
3D model fits best with the tracked points (black lines). The 
current visual features are the projection of the 3D lines U 
according to the pose k M 0 and the desired visual features 
are the tracked points pi. The error is the distance between 
a point and a line (see bottom right frame Fig. 3). 

Finally the optimisation problem can be written as: 

k M 0 = arg min V"' C(d±(pi,li( k M 0 ))) (13) 

k M a i 

where C is a robust function that allows to handle outliers. 
The distance d± is represented in Fig. 3. 


model projection model projection 

at time t ,at time 1+1 



Fig. 3. Model-based tracking principle. 


The robust part of the tracker can not be found in the 
features extraction itself but in the weighting of their contri¬ 
butions relatively to the confidence given in each measure¬ 
ment. Classically, the outliers are rejected using Hough or 
RANSAC methods. The considered tracker is based on a 
statistical methods, the M-Estimator [18]. Further details on 
the algorithm for robust tracking may be found in [17]. 
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B. Visual Servoing 

Classically, visual servoing aims to regulate an error vector 
e = s — s* between some current features observed in an 
image s and some desired visual features 8* [19]. 

Here the current and the desired features are the current 
pose of the object in the camera frame k M 0 and the pose of 
the object in the desired camera frame k * M 0 . The positioning 
task is regulated when k M\ = I. The task error can be 
expressed as 6 dimensional pose vector 8 = (£, 6u)\ the first 
three coordinates are the three translations t and the last three 
coordinates are a rotation vector in a (6u) representation 
where Ou defines the angle and axis of the rotation of the 
current camera with regards to the desired one. 

The key feature in this control scheme is the interaction 
matrix L that links the time variation of the visual features 
s to the relative camera velocity k. It is defined by: 

s = Lk (14) 


Then, the control law that regulates e with an exponential 
decrease e = — Ae is [19]: 

k = -A L+e (15) 


where L+ denotes the Moore-Penrose pseudo inverse of an 
approximation or a model of L. 

For the chosen type of features, the interaction matrix is 
[19] 


( k *R k 0 3x3 ) 

\03x3 Juj J 


(16) 


where J u = L u and L u is such that L UJ 1 9u = 6u. 

Then we can write the CoM reference speed c : 

t = -X c v k L+e (17) 


C. Cancelling the sway motion 

Due to the sway, the features oscillate in the image. Using 
(12) and (15) the feature variations can be written: 

s = Lk + L k V c b (18) 

Let us define a virtual camera (Fig. 4) /C that corresponds 
to the position of the on-board camera if there was no sway 
motion. The velocity of this virtual camera is k, it is actually 
the velocity that is input into the reactive PG. Its value is 
given in(23). In order to compute a control law that does not 
include the sway motion, we will servo this virtual camera 
s(k) to s(k*). 




K 



t=0 




t=T 


o 






Fig. 4. /C is the current camera frame and 1C is the camera position obtained 
if the visual servoing velocity is applied without the walking constraints. 


We have now to express s = s{k ) with regards to the 
current measurement s = s(k). With (14) we can write: 

s(t) — 8(0) = f Lkdt = f L(k + bk)dt (19) 
J o Jo 

and s(t) — 8(0) = / Lkdt (20) 

Jo 

Then assuming that 8(0) — 8(0) = E and using (19) and 
(20) we obtain 

s(t) = s(t) + f L k V c bdt - E (21) 

Jo 

from which we can deduce the corrected visual error 


e(t ) = s(t) - s* = e(t ) - ( [ L k V c bdt - E) 

Jo 


( 22 ) 


Notice that when e —» 0 then e —> f Q Lbk . In this study, 
we do not expect e to converge to zero but to oscillate around 
zero with a period T. The convergence of the control law 
is then reached when f*_ T edt = 0, which is obtained if 

ft-T Jo L k V c bdt = 0 . Let us define E = f*_ T L k V c bdt 
and note that in general E ^ 0. It can be estimated over one 
period of time T. In order to avoid drift accumulation in the 
comutation of E we can use a sliding windows to define the 
current virtual error e and deduce the control law 

k = -\L + (e-([ L k V c bdt- [ [ L k V c bdt )) (23) 

Jo Jt-T Jo 

And finally, the CoM reference velocity can be computed 
as : 


c=- X 


'V k L+{e-{[ L k V c bdt- [ [ L k V c bdt )) 

Jo Jt-T Jo 


(24) 


D. Vision based control 

Visual Model Predictive Control Scheme has been studied 
to deal with constraints, eg to ensure the visibility of the 
target or avoid joint limits [20]. In order to improve the 
results presented in this paper, we propose to write a general 
non linear model predictive control scheme to select the 
optimal jerk of the CoM C regarding some visual criteria. 
Then the function to minimise is now 

1 N 

min - ^||s(fci)-s*f (25) 

° i=1 

IV. A PRELIMINARY RESULT 

Fig. 5, Fig. 6 depicts an experiment of visual servoing for 
dynamic walking that shows the feasibility of the approach 
on the HRP2 robot. 

The experimental scheme is the following : the robot has 
to reach a desired position with regards to a partially known 
object. First, the system is given a desired pose of the object 
in the camera frame. It can be arbitrarily set or it can be 
estimated by placing the robot at the desired position. The 
object is then tracked in the image and its pose is estimated 
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Fig. 5. Model tracking while walking on a square on the ground. The robot firstly walks forward, then sideways, then backwards, and sideways again 
to reach its initial position. On images 2-8, we can notice white horizontal curved line. They are induced by the reflexion of the light on the dark plastic 
shield. The model-tracker gives good positioning results when the tracked object has 3 dimensional edges. 


in the camera frame. It allows to learn k*M 0 . The advantage 
of the learning solution is that the position is estimated using 
the same camera as the one used on-line for visual servoing, 
which compensate for calibration errors. Secondly the robot 
is placed to an initial arbitrary position k M 0 from where the 
reference object can be seen. In this paper, the robot has to 
perform both translational motions and rotational motion to 
reach the desired position. 

Fig. 5 presents some tracking results while the robot is 
walking. On images 2-8, we can notice white horizontal 
curved line. They are induced by the reflexivity of the light 
on the dark plastic shield that protects the HRP-2 camera. 
They cause some partial occlusion and worse, they can 
masquerade object lines and make the tracking fail. The 
model-based tracker we use is designed for convex objects. 
If the current projection of the model only allows to track 
2D planes, it can happen that the optimisation problem falls 
in a local minimum. The order of magnitude of the model¬ 
tracking accuracy, while walking 2m away from the object, 
is about 0.1m and O.lrad 

In the experiment illustrated Fig. 6 a position based visual 
servoing is given the Pattern Generator as an input. In this 
experiment, the robot has approximatly to move 1 m forward, 
1.5m sideways and 0.7 rad in rotation. A convergence thresh¬ 
old is arbitrarily set to 0.1m in translation and O.lrad in 
rotation. Then the accuracy of the positioning reaches these 
values at best. Besides, the reference velocity is limited to 
0.2m/,s in translation and 0.2 rad/s in rotation. This limits 
have be chosen to secure the robot mechanical parts. It may 
be increase in the future. 

The top left figure depicts the estimated position of the 
object frame with regards to the camera frame. Notice that 
the on board camera axis are not parallel to the ground plane. 
The robot head is oriented slightly towards the ground. Also 
remark that the position estimation could be replaced by any 
localisation technique (such as SLAM) except that the model 
based scheme has the advantage to handling mobile object 
tracking 1 . In the top left graph, we can see that the lateral 

1 some examples of rolling ball tracking and other objects tracking are 
available on the lagadic team website : http://www.irisa.fr/lagadic/demo.html 


motion oscillates. This is directly due to the stepping motion. 
This lateral motion can be also found in the bottom right 
figure that is the output velocity of the pattern generator. The 
walking control guaranties this sway motion to be minimal. 
Furthermore, the upper body of the HRP2 robot can not 
compensate for this sway motion due to a lack of degrees 
of freedom. Anyway, the model tracker proved to be robust 
enough to track an object even when the camera oscillates 
under the sway motion. The top right figure first shows an 
increase of the error and then a visual servoing classical 
exponential decrease. The increase of the error is directly 
related to the variation of the pose estimation that can be 
observed in the top left figure. Both changes are due to the 
motion induced by the robot first steps. Usually, the robot 
needs two steps to reach the desired velocity and make the 
error decrease. The bottom left figure presents the visual 
control law that is the reference pattern generator velocity. 

The resulting motion has been compared to a planned 
trajectory executed with a Kajita’s PG. As shown in [21], 
when the robot is walking forward or backward, the open 
loop execution of the planned trajectory results in a good 
positioning (less than 1cm in translation and less than 0.1 rad 
in rotation). However, lateral motion induces a large drift 
and an error in rotation, such that the difference between 
the initial position and the final position is more than 60cm 
in translation and more that 0.5ra<i in rotation. Since we 
have set the convergence threshold to 0.1, the translational 
error is less than 2cm. Yet, the error in rotation is small and 
less than 0.1 rad even after lateral motion. The difference 
between the first position and the final position was less that 
15cm and the error in rotation less than O.lrad As excepted, 
the greater error are found along the sagital plane. Indeed, 
this is the direction where the pose estimation is the more 
uncertain for a monocular camera. 

V. Conclusion 

We think that vision based control is well suited to control 
the walk of the humanoid robot HPR-2. The method proved 
to be robust to model errors and gives better result than 
executing a planned trajectory without closing the control 
loop. Our going work on vision based pattern generator 
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Fig. 6. Visual servoing of dynamic walking experiment. Top left : trajectory of the object’s pose in the camera frame. Top right : evolution of the error 
norm. Bottom left : control Input of the pattern generator (reference CoM velocity c). Bottom right : control output of the pattern generator (real CoM 
velocity c), we can remark two picks to zero which are only due to client reading error from the middle ware and these values are not the one sent to the 
system 


is expected to improve these results in several ways : the 
predictive framework allows to include both balance and 
visibility constraints, we expect the system to be more 
reactive and we expect more natural trajectories that do not 
necessary follows an exponential decrease of the error. 
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